Multilinear Spectral Regularization for Kernel-based Multitask Learning

Recent research in machine learning witnessed a renewed interest in tensors. In particular, multilinear algebra has been leveraged to derive structured finite dimensional parametric models [1, 2]. In [3] these ideas have been generalized to reproducing kernel Hilbert spaces. The arising framework comprises existing problem formulations, such as tensor completion [4], as well as novel functional formulations. The approach is based on a class of regularizers for tensor product functions, termed multilinear spectral penalties, that is related to spectral regularization for operator estimation [5]. In this work we outline the main ideas and focus on the implications for (multilinear) multitask learning. Multi-task Learning (MTL) aims at simultaneously finding multiple predictive models, each of which corresponds to a learning task. In many cases of interest MTL has been shown to improve over the case where tasks are learned in isolation, see [6] and references therein. Importantly, the approach allows one to make predictions even in absence of training data for one or more of the tasks. Therefore it is suitable to perform transfer learning [7]. Recently [2] has proposed an extension, termed Multilinear Multi-task Learning (MLMTL), to account for multi-modal interactions between the tasks. This is a departure from classical tensorbased methods, where the multilinear decomposition is performed on the input data.